Method for Accurately Determining the Position and Orientation of Each of a Plurality of Identical Recognition Target Objects in a Search Target Image

ABSTRACT

Embodiments of the invention relate to detecting the number, position, and orientation of objects when a plurality of recognition target objects are present in a search target image. Dictionary image data is provided, including a recognition target pattern, a plurality of feature points of the recognition target pattern, and an offset (O x , O y ) from the coordinates at the center of the image to the coordinates of the feature point. The sizes (R t ) and directions (θ t ) of feature vectors for the coordinates (T x , T y ) of a plurality of feature points in the target image are also provided. The coordinates (F x , F y ) of a virtual center point in the target image is derived. Additional virtual center points within a radius of the coordinates (F x , F y ) is counted. Presence of a recognition target object is recognized near the virtual center point coordinates of the search target image.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation patent application claiming thebenefit of the filing date of U.S. patent application Ser. No.14/806,095 filed on Jul. 22, 2015 and titled “Method for AccuratelyDetermining the Position and Orientation of Each of a Plurality ofIdentical Recognition Target Objects in a Search Target Image” nowpending, which is hereby incorporated by reference, and which claimspriority under 35 U.S.C. §119 from Japan Patent Application No.2014-156930, filed on Jul. 31, 2014, the entire contents of which areincorporated by reference herein.

BACKGROUND

The present invention relates to the recognition of objects in images.More specifically, the embodiments relates to detecting the number,position, and orientation of each recognition target object when aplurality of recognition target objects are present in a search targetimage.

In recent years, object recognition using rotation- and scale-invariantlocal image features known as keypoints has reached the practical stage.See Lowe, “Distinctive Image Features from Scale-Invariant Keypoints”2004, hereinafter referred to as Lowe, and Rublee et al., “ORB: AnEfficient Alternative to SIFT or SURF” 2011, hereinafter referred to asRublee et al.

In the technique described in Lowe, known as SIFT, Gaussian filters withdifferent spatial scales are used on images. Differences in the outputfrom filters with adjacent scales are extracted, and image sets known as“Difference of Gaussians” (DoG) are obtained. Coordinates at which theabsolute values in a DoG image are at their maximum in both the spatialdirection and scale direction are called keypoints, and a plurality ofkeypoints are usually detected in an image with shading patterns. Theorientation of the keypoints is determined from the density gradient ofthe pixels surrounding the keypoints, and the maximum scale of the DoGis used as the keypoint scale. The pixels surrounding keypoints aredivided into 16 blocks, and a shading histogram of the pixels in eachblock is extracted for use as a feature value of the keypoints.

In SIFT, feature values are expressed as 128-dimensional vectorsincluding a real number element. In the technique described in Rublee etal., known as oFAST, corner portions of shading patterns of pixels areused as keypoints. As in the case of SIFT, oFAST uses both scale anddirection. FIGS. 1A and 1B are diagrams showing an example of keypointdetection using the oFAST method of the prior art. In this detectionexample, the image in FIG. 1A is a dictionary image, and the patterns inthe dictionary image are the recognition target objects. The detectedkeypoints are shown in FIG. 1B. The circle (o) containing cross-hatchingare the detected keypoints. The techniques described in both Lowe andRublee et al. are able to determine with great accuracy whether or notthe recognition target objects in a dictionary image are search targetimages.

However, in situations such as unrecognition of the store products,there may be a plurality of objects present in the search target image.In such situations, the existing techniques can determine whether or notrecognition target objects are present. However, the number ofrecognition target objects, and the position and orientation of eachobject in the search target image cannot be determined.

SUMMARY

The aspects include a method for recognition of one or more objects inan image.

In one aspect, the method is a computer implemented method forrecognition of objects in an image, dictionary image data with arecognition target pattern is provided. The data includes feature pointsof the recognition target pattern, including a size (R_(m)) anddirection (θ_(m)) of a feature vector, and an offset (O_(x), O_(y)) fromcoordinates at a center of a target image to coordinates of a featurepoint. In addition, a size (R_(t)) and direction (θ_(t)) of the featurevector for coordinates (T_(x), T_(y)) of a plurality of feature pointsin the target image are provided. Coordinates (F_(x), F_(y)) of avirtual center point in the target image derived from T_(x), T_(y),O_(x), O_(y), R_(m), R_(t), θ_(m), and θ_(t), are calculated. Inaddition, a number of additional virtual center points within apredetermined radius (r) of the coordinates (F_(x), F_(y)) of thevirtual center pointare counted. The coordinates (F_(x), F_(y)) of thevirtual center point and the number of counted virtual center points arestored.

Other features and advantages will become apparent from the followingdetailed description of the presently preferred embodiment(s), taken inconjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The drawings referenced herein form a part of the specification.Features shown in the drawings are meant as illustrative of only someembodiments of the invention, and not of all embodiments of theinvention unless otherwise explicitly indicated.

FIGS. 1A and 1B are diagrams showing an example of keypoint detectionusing the oFAST method of the prior art. FIG. 1A is a dictionary imageand FIG. 1B shows the detected keypoints.

FIG. 2A is a diagram showing the results of detecting a recognitiontarget object from the detected keypoints in the method of the priorart.

FIG. 2B is a diagram showing a circle centered on the keypointcenter-of-gravity coordinates.

FIG. 3 is a diagram showing the results of detecting a plurality ofrecognition target objects from a search target image in the method ofthe prior art.

FIG. 4 is a flow chart illustrating the process for converting keypointcoordinates to virtual center point coordinates.

FIG. 5 is a diagram showing a keypoint in a dictionary image when thepresent invention is applied to coordinate conversion.

FIG. 6 is a diagram showing a matching point and its virtual centerpoint in a search target image when the present invention is applied tocoordinate conversion.

FIG. 7A is a diagram showing the keypoints in a dictionary image.

FIG. 7B is a diagram showing the virtual center point of the searchtarget image resulting from the application of the present invention.

FIG. 8 is a diagram showing the results of applying the presentinvention when a plurality of recognition target objects are present ina search target image.

FIG. 9 is a diagram showing the results of applying the presentinvention when a plurality of recognition target objects are adjacent toeach other.

FIG. 10 depicts a block diagram of a computer system and associatedcomponents for implementing an embodiment.

DETAILED DESCRIPTION

It will be readily understood that the components of the presentinvention, as generally described and illustrated in the Figures herein,may be arranged and designed in a wide variety of differentconfigurations. Thus, the following detailed description of theembodiments of the apparatus, system, and method of the presentinvention, as presented in the Figures, is not intended to limit thescope of the invention, as claimed, but is merely representative ofselected embodiments of the invention.

Reference throughout this specification to “a select embodiment,” “oneembodiment,” or “an embodiment” means that a particular feature,structure, or characteristic described in connection with the embodimentis included in at least one embodiment of the present invention. Thus,appearances of the phrases “a select embodiment,” “in one embodiment,”or “in an embodiment” in various places throughout this specificationare not necessarily referring to the same embodiment.

The illustrated embodiments of the invention will be best understood byreference to the drawings, wherein like parts are designated by likenumerals throughout. The following description is intended only by wayof example, and simply illustrates certain selected embodiments ofdevices, systems, and processes that are consistent with the inventionas claimed herein.

The invention disclosed in the present application proposes a techniquefor detecting the number, position, and orientation of each recognitiontarget object when a plurality of recognition target objects are presentin a search target image.

The methods in the existing means used to detect recognition targetobjects in search target images perform the following steps.

First, keypoints and their feature vectors are obtained from adictionary image containing a recognition target object, and keypointsand their feature vectors are obtained from the search target image inthe same manner. Next, a brute-force method is used to determine thedistance between feature vectors on the keypoints in the two images, andto determine pairs of keypoints closest to each other whose separatingdistance does not exceed a threshold value. These pairs of keypoints arecalled keypoint pairs.

FIGS. 2A and 2B are diagrams showing the results of detecting arecognition target object from the detected keypoints in the method ofthe prior art. When the number of keypoint pairs exceeds a certainthreshold value, the center-of-gravity coordinates of the keypoints inthe search target image forming keypoint pairs are determined fromkeypoints detected in the manner shown in FIG. 1B. The circle depictedusing bold lines in FIG. 2B is a circle centered on the keypointcenter-of-gravity coordinates.

It is then determined that there is a recognition target object aroundthe coordinates. One problem associated with existing means is that thecenter-of-gravity coordinates are biased in areas with a highconcentration of keypoints so keypoints are not necessarily distributeduniformly over recognition target objects. As a result, the center ofthe dictionary image does not match up with the center-of-gravitycoordinates. Thus, these methods cannot be used in applicationsrequiring object position accuracy, such as marking by inkjets, andobject picking by robots.

Another problem associated with prior art is that the number, position,and orientation of objects cannot be detected simply by determining thecenter-of-gravity coordinates of keypoints when a plurality ofrecognition target objects are present in a search target area. FIG. 3is a diagram showing the results of detecting a plurality of recognitiontarget objects from a search target image in the method of the priorart. In the detection results, there are two circles depicted using boldlines for four recognition target objects, and adjacent recognitiontarget objects cannot be differentiated. Methods differentiating objectsby limiting the keypoint search range have been considered, but thekeypoint search area has to be the same size as the recognition targetobjects. This makes it difficult to differentiate adjacent recognitiontarget objects when identifying store products (FIG. 3).

The present embodiment(s) proposes a process for converting keypointcoordinates to virtual center point coordinates. Referring to FIG. 4, aflow chart (400) is provided demonstrating the process for convertingkeypoint coordinates to virtual center point coordinates. FIG. 5 is adiagram showing a keypoint in a dictionary image when the presentinvention is applied to coordinate conversion. The shaded circles (o)are keypoints.

The scale R_(m), direction θ_(m) [radian], and offset (O_(x), O_(y))from the center of the dictionary image of the keypoints of thedictionary image are stored (402). All of the parameters can be storedin the memory of the computer serving as the hardware resource in theform of a sequence in the computer program serving as the softwareresource. The scale of a keypoint is proportional to the size of theobject, see FIG. 5. In one embodiment, at step (402) the feature vectorof each keypoint is determined and stored.

Similarly, the coordinates (T_(x), T_(y)), scale R_(t), and directionθ_(t) [radian] of the keypoints in the search target image are stored(404). Also, the feature vector of each keypoint is determined. (406) Abrute-force method is used on the keypoints in the dictionary image andthe search target image to determine the distance between featurevectors (408), and to determine pairs of keypoints closest to each otherwhose separating distance does not exceed a threshold value (410). Thesepairs are referred to as keypoint pairs.

FIG. 6 is a diagram showing a matching point and its virtual centerpoint in a search target image when the present invention is applied tocoordinate conversion. The coordinates (F_(x), F_(y)) of thecorresponding virtual center point are defined and calculated for all ofthe keypoint coordinates (T_(x), T_(y)) in the keypoint pairs (FIG. 6)(412) using the following equations:

F _(x) =T _(x)+(O _(x) ·R _(t) /R _(m)) (cos(θ_(t)−θ_(m)))+(O _(y) ·R_(t) /R _(m)) (sin(θ_(t)−θ_(m)))

F _(y) =T _(y)+(O _(x) ·R _(t) /R _(m)) (−sin(θ_(t)−θ_(m)))+(O _(y) ·R_(t) /R _(m)) (cos(θ_(t)−θ_(m)))

As shown in FIG. 6, the shaded circle (o) is a keypoint, and thecross-hatched circle (o) is the virtual center point.

The number of additional virtual center points within a predeterminedradius (r) of the virtual center point coordinates (F_(x), F_(y)) iscounted (414), and the count is stored (416). The process shown in steps(414) and (416) is executed on all of the virtual center points, and thevirtual center point coordinates (M_(x), M_(y)) with the highest countare stored as a candidate indicating the possibility of a match with thecenter of the corresponding dictionary image (418). When the number ofcounted virtual center points exceeds a predetermined threshold value(N_(c)) (420), it is determined that a recognition target object presentin the dictionary image is near the virtual center point coordinates(M_(x), M_(y)) of the search target image (422). The presentembodiment(s) is designed so that, when the search target image includesa recognition target object of the dictionary image, the coordinates ofthe virtual center point groups of the keypoint pairs are concentratednear the center of the dictionary image.

The average value of the difference θ_(t)−θ_(m) between the directionθ_(t) [radian] of the keypoints surrounding (M_(x), M_(y)) and thedirection θ_(m) [radian] of the keypoints of the correspondingdictionary image is the orientation of the recognition target object inthe search target image (FIGS. 7A and 7B). More specifically, FIG. 7A isa diagram showing the keypoints in a dictionary image, and FIG. 7B is adiagram showing the virtual center point of the search target imageresulting from the application of the present invention. In FIG. 7B, thecenter of the circle with the bold line is (M_(x), M_(y)). Also, asshown in FIG. 7B, the direction of the line with the shadedcross-hatching is the orientation of the recognition target object. Thedirection of the line shown with the cross-hatching indicates the 1200(0000) direction of a clock hand from the center (M_(x), M_(y)) of thecircle with the bold line.

FIG. 8 is a diagram showing the results of applying the presentinvention when a plurality of recognition target objects are present ina search target image. When the search target image includes a pluralityof recognition target objects, the virtual center point group isconcentrated on a plurality of coordinates (FIG. 8). The number is thenumber of recognition target objects, and each group of virtual centerpoint coordinates (M_(x), M_(y)) corresponds to the center of adictionary image. When the search target image includes a plurality ofrecognition target objects, the center of each circle is the center of arecognition target object.

FIG. 9 is a diagram showing the results of applying the presentinvention when a plurality of recognition target objects are adjacent toeach other. The number of recognition target objects and the virtualcenter point coordinates (M_(x), M_(y)) of each recognition targetobject can be correctly determined even when a plurality of recognitiontarget objects are adjacent to each other (FIG. 9). When a plurality ofrecognition target objects is adjacent to each other, the center of eachcircle is the center of a recognition target object.

The characteristics of the technical idea of the present inventionexplained above can be realized as a technique (method, system, computerprogram) executed by a computer. All of the operations, includingproviding dictionary image data, calculating the coordinate conversion,counting the virtual center points, and storing the counted number, canbe executed by a computer. Extraction of keypoints, representation ofthe feature values as vectors, and the storage of these can be performedvery efficiently using a computer. The present invention demonstratesaccurately calculating the position and orientation of each of aplurality of recognition target objects when a plurality of identicalrecognition target objects are present in a search target image.

Referring now to the block diagram of FIG. 10, additional details arenow described with respect to implementing an embodiment. The computersystem includes one or more processors, such as a processor (1002). Theprocessor (1002) is connected to a communication infrastructure (1004)(e.g., a communications bus, cross-over bar, or network).

The computer system can include a display interface (1006) that forwardsgraphics, text, and other data from the communication infrastructure(1004) (or from a frame buffer not shown) for display on a display unit(1008). The computer system also includes a main memory (1010),preferably random access memory (RAM), and may also include a secondarymemory (1012). The secondary memory (1012) may include, for example, ahard disk drive (1014) and/or a removable storage drive (1016),representing, for example, a floppy disk drive, a magnetic tape drive,or an optical disk drive. The removable storage drive (1016) reads fromand/or writes to a removable storage unit (1018) in a manner well knownto those having ordinary skill in the art. Removable storage unit (1018)represents, for example, a floppy disk, a compact disc, a magnetic tape,or an optical disk, etc., which is read by and written to by removablestorage drive (1016).

In alternative embodiments, the secondary memory (1012) may includeother similar means for allowing computer programs or other instructionsto be loaded into the computer system. Such means may include, forexample, a removable storage unit (1020) and an interface (1022).Examples of such means may include a program package and packageinterface (such as that found in video game devices), a removable memorychip (such as an EPROM, or PROM) and associated socket, and otherremovable storage units (1020) and interfaces (1022) which allowsoftware and data to be transferred from the removable storage unit(1020) to the computer system.

The computer system may also include a communications interface (1024).Communications interface (1024) allows software and data to betransferred between the computer system and external devices. Examplesof communications interface (1024) may include a modem, a networkinterface (such as an Ethernet card), a communications port, or a PCMCIAslot and card, etc. Software and data transferred via communicationsinterface (1024) is in the form of signals which may be, for example,electronic, electromagnetic, optical, or other signals capable of beingreceived by communications interface (1024). These signals are providedto communications interface (1024) via a communications path (i.e.,channel) (1026). This communications path (1026) carries signals and maybe implemented using wire or cable, fiber optics, a phone line, acellular phone link, a radio frequency (RF) link, and/or othercommunication channels.

In this document, the terms “computer program medium,” “computer usablemedium,” and “computer readable medium” are used to generally refer tomedia such as main memory (1010) and secondary memory (1012), removablestorage drive (1016), and a hard disk installed in hard disk drive(1014).

Computer programs (also called computer control logic) are stored inmain memory (1010) and/or secondary memory (1012). Computer programs mayalso be received via a communication interface (1024). Such computerprograms, when run, enable the computer system to perform the featuresof the present embodiment(s) as discussed herein. In particular, thecomputer programs, when run, enable the processor (1002) to perform thefeatures of the computer system. Accordingly, such computer programsrepresent controllers of the computer system.

The present embodiment(s) may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent embodiment(s).

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations maybe assembler instructions, instruction-set-architecture (ISA)instructions, machine instructions, machine dependent instructions,microcode, firmware instructions, state-setting data, or either sourcecode or object code written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Smalltalk, C++ or the like, and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The computer readable program instructions mayexecute entirely on the user's computer, partly on the user's computer,as a stand-alone software package, partly on the user's computer andpartly on a remote computer or entirely on the remote computer orserver. In the latter scenario, the remote computer may be connected tothe user's computer through any type of network, including a local areanetwork (LAN) or a wide area network (WAN), or the connection may bemade to an external computer (for example, through the Internet using anInternet Service Provider). In some embodiments, electronic circuitryincluding, for example, programmable logic circuitry, field-programmablegate arrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present embodiment(s).

Aspects of the present embodiment(s) are described herein with referenceto flowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments. Itwill be understood that each block of the flowchart illustrations and/orblock diagrams, and combinations of blocks in the flowchartillustrations and/or block diagrams, can be implemented by computerreadable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowcharts and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe functions/acts specified in the flowcharts and/or block diagramblock or blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowcharts and/or block diagram block orblocks.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present invention has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the invention. Theembodiment was chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

It will be appreciated that, although specific embodiments of theinvention have been described herein for purposes of illustration,various modifications may be made without departing from the spirit andscope of the invention. Accordingly, the scope of protection of thisinvention is limited only by the following claims and their equivalents.

We claim:
 1. A computer implemented method comprising: providingdictionary image data including a recognition target pattern, aplurality of feature points of the recognition target pattern includinga size (R_(m)) and direction (θ_(m)) of a feature vector, and an offset(O_(x), O_(y)) from coordinates at a center of a target image tocoordinates of a feature point; providing a size (R_(t)) and direction(θ_(t)) of the feature vector for coordinates (T_(x), T_(y)) of aplurality of feature points in the target image; calculating coordinates(F_(x), F_(y)) of a virtual center point in the target image derivedfrom T_(x), T_(y), O_(x), O_(y), R_(m), R_(t), θ_(m), and θ_(t);counting a number of additional virtual center points within apredetermined radius (r) of the coordinates (F_(x), F_(y)) of thevirtual center point; and storing the coordinates (F_(x), F_(y)) of thevirtual center point and the number of counted virtual center points. 2.The method of claim 1, further comprising repeating the counting andstoring on all feature points in the target image.
 3. The method ofclaim 1, wherein each feature point in the plurality of feature pointsin the target image is a matching feature point of at least one featurepoint in the target image and at least one feature point in therecognition target pattern.
 4. The method of claim 2, wherein repeatingthe counting and storing on all feature points in the target imageincludes converting the coordinates (F_(x), F_(y)) of the virtual centerpoint of a maximum number of counted virtual center points to arecognition target image center.
 5. The method of claim 5, furthercomprising establishing the coordinate (F_(x), F_(y)) as one recognitiontarget image center and eliminating all of the virtual center pointswithin a predetermined radius (r) of the coordinates (F_(x), F_(y)). 6.The method of claim 1, wherein calculating the coordinates of thevirtual center point in the target image uses the following equations:F _(x) =T _(x)+(O _(x) ·R _(t) /R _(m)) (cos(θ_(t)−θ_(m)))+(O _(y) ·R_(t) /R _(m)) (sin(θ_(t)−θ_(m)))F _(y) =T _(y)+(O _(x) ·R _(t) /R _(m)) (−sin(θ_(t)−θ_(m)))+(O _(y) ·R_(t) /R _(m)) (cos(θ_(t)−θ_(m)))